Objective study of the performance degradation in emotion recognition through the AMR-WB+ codec

Research in speech emotion recognition often involves features that are extracted in lab settings or scenarios where speech quality is high. However, a great deal of communication occurs through speech codecs, which alters the speech signal and features extracted from it. The purpose of this study is to report on the performance degradation in emotion recognition systems when speech is passed through a codec and to provide insight on features that are affected in relation to their relevance in emotion classification. Using two emotional databases and the AMR-WB+ codec, features that are the most and least significantly affected by the codec are investigated and classifier performances are compared among them in multiple experiments. The results show that clean-trained classifiers drop significantly in accuracy on codec speech, and vice versa for codec-trained classifiers on clean speech in a full feature set task. However, using an intersection feature set between two databases that is resilient to the codec process can provide comparable performance for clean and codec-trained classifiers on either type of speech. The results suggest that these sets of features seem to capture more relevant information about emotion classes, since the perception of emotion should not be altered by a codec.